Tracking a Single Pixel

Nd787 C4 L06 04 Tracking A Single Pixel V1

Math Walkthrough

The diagram below shows a pixel that has moved from (x,y) at time t to (x+u, y+v) at time t+1.

From this picture, it's easy to figure out the velocity vector (u,v). But when we look at two real images, we'd first need to solve what's called the pixel correspondence problem. That is, we need to know which pixels in image 2 correspond to which pixels in image 1.

To solve this problem we make two assumption.

Assumption 1: the motion is small: this means we can look in the vicinity of where the pixel was to try to determine where it now is.

Assumption 2: the appearance doesn't change from t to t+1: this assumption is best expressed mathematically.

I(x,y,t) = I(x+u, y+v, t+1)

This relationship is known as the brightness constancy constraint.

If we drop the time index and do a Taylor series expansion of the right hand side of this equation, we find the following:

I(x+u,y+v) \approx I(x, y) + \frac{\partial I}{\partial x} u + \frac{\partial I}{\partial y} v

We can plug this back into the brightness constancy constraint and reorganize some terms to find the following:

\frac{\partial I}{\partial t} + \frac{\partial I}{\partial x} u + \frac{\partial I}{\partial y} v = 0

Or using the notation from the video (which is common in this field):

I_t + I_x u + I_y v = 0

This equation says that any change in the appearance of a pixel over time has to be explained by spatial motion induced by the camera movement.

Note: I(x,y,t) gives the intensity of the pixel at location (x,y) at time t. If this is a gray-scale image, this intensity would be a single number corresponding to the darkness of the pixel. If this were a color image it might be a 3-vector with values giving the amount of red, green, and blue in the image.

Next Concept